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ABSTRACT 



Students' academic achievement scores have been found 



to improve with the use of graphic organizers. Unfortunately, most 
previous research on graphic organizers has been constrained to 
English speakers in secondary and higher education. This study 
examines the effectiveness of graphic organizers when used by fourth 
grade students engaged in computer based instruction. Additionally, 
whether the learners 's dominant language (Spanish or English) 
influenced the effectiveness of graphic organizers. Immediate and 
delayed gain in academic achievement scores of all students was 
analyzed. Statistically and educationally significant differences 
were found in scores favoring students who used graphic organizers. 
Results also indicated that when using computer-based instructional 
programs with embedded graphic organizers, Spanish and English 
speaking students, as groups, did not vary on learning style to the 
point of significantly Impacting academic achievement scores 
according to tests measuring short-term and long-term recall. 
Limitations of the study and suggestions for future research are 
examined. Concepts are illustrated in two figures, and data is 
summarized in three tables. (Contains 23 references.) (Author /MAS) 
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Abstract 



Students' academic achievement scores have been found to improve with the use of graphic organizers. 
Researchers suggest this may be due to the way graphic organizers depict concepts and relationships 
between concepts. Unfortunately, most previous research on graphic organizers has been constrained to 
English speakers in secondary and higher education. To expand our knowledge on graphic organizers and 
increased learning, the effectiveness of graphic organizers when used by fourth-grade students engaged in 
computer-based instruction was examined. Additionally, whether the learners' dominant language (Spanish 
and English) influenced the effectiveness of graphic organizers was studied. A ond-way analysis of variance 
was used to analyze immediate and delayed gain in academic achievement scores of all students. Statistically 
and educationally significant differences were foimd in scores favoring students who used graphic organizers 
. over those who used lists of topics. A two-way analysis of variance identified no significant differences 
between language groups for the effect of graphic organizers on immediate or delayed tests. Limitations of 
the study and suggestions for future research are examined. 
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Background 



Intrc>d\iCtiQn , , j 

Cognitive psychologists generally agree that there are at least two disUnct types of human knowledge -- 
declarative and procedural (Anderson, 1983; Jonassen, Beissner, & Yacci, 1993). Declarative knowledge is 
knowing what something is, but it does not imply knowing how to use rliat knowledge. Defining the word 
"computer" and identifying the number 24 on a number line are examples of declarative knowledge. 
Procedural knowledge is knowing how to do something. Knowing how to manipulate physical entities such 
as accessing a computer application, or mental manipulations such as subtracting two-digit numbers, are 
examples of procedural knowledge. Procedural knowledge is dependent on an awareness of an object or idea 
(declarative knowledge) even though we may not be able to articulate that knowledge (Jonassen et al., 
1993). 

But how does declarative knowledge evolve to the complex forms required for procedural knowledge? To 
help explain this transition, some psychologists propose an intermediate knowledge type in which facts and 
concepts learned as declarative knowledge are interrelated with one another to form complex systems. This 
interrelationship of information, and the way in which we mentally organize it, has been called structural 
ibwM'/edge (Diekhoff, 1983). . ... u 

Sa-uctural knowledge can be thought of as a network of mental connections or relaUonshsps between pieces 
of declarative knowledge. As learners develop these structures, they more easily associate independent ideas. 
These connections allow learners to draw conclusions and understand relationships among concepts. As 
students begin to develop these underlying structures and organizations of declarative knowledge, they go 
beyond "knowing what" to a better understanding of "knowing why." Structural knowledge networks appear 
to create the interconncctedness of ideas that support the development of procedural knowledge. 
A variety of techniques have been developed to elicit, represent, and convey structural knowledge (Jonassen 
et al., 1993). One metliod being employed in public education is graphic organizers. Graphic organizers are 
spatial metaphors that indicate relationships among concepts in a node-link-node visual display (Anderson, 
1990; Jonassen, 1990; Jonassen et al., 1993). Nodes contain key concepts. Links depict unspecified 
relationships between nodes (see Figure 1). 
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Figure 1 . Sample graphic organizer. 

Graphic organizers convey relationships and content structures in a pictorial fashion. These visual 
representations provide learners with a structural overview of information to be learned. This overview 
directs learner's attention towards key concepts and conceptual relationships rather than seemingly isolated 
facts. The use of graphic organizers enhances the understanding, organization, and long-term retention of 
information (Stevensokl & Wilson, 1990) and accentuates meaningful learning and information 
manipulation (Jonassen, 1990; Kerchner. 1990; Peel. 1992). Graphic organizers also facilitate the 
extrapolation, combination, inference, and other logical reasoning mechanisms that allow learners to 
transfer and apply information (Jonassen et al., 1993). 
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Grapllic organizers improve the effectiveness of instruction. However, it cannot be assumed tiiat they affect 
all students equally. In fact, researchers suggest that the effectiveness of any instructional strategy varies 
with individual learner differences (Dunn, 1990; Oxford 1990; Scarpaci & Fradd, 1985). 
Individual learner differences include any personal trait or capability that influence the way students process 
and use information. Although there are a variety of differences that affect learning, dominant language is a 
key difference because it ser/es as a central means for information processing (Sticht, 1992). Dominant 
language differences do not imply a difference in intelligence, but rather in the way individuals organize and 
use information (Scarpaci Sc Fradd, 1985). 

Previous research on the effectiveness of graphic organizers generally focused on Anglo-Americans in 
secondary and higher education. In a review of the literature, no studies were identified that included a 
bilingual (Spanish - English) elementary school-aged population engaged in computer-based instruction. 
This omission limits the gencralizability of previous research findings as to the effectiveness of graphic 
organizers to various populations. 

Purpose of Study 

The purpose of this study was two-fold: (a) to assess the effectiveness of graphic organizers in computer- 
based instruction for fourth-grade students, and (b) to determine the relationship between learner's dominant 
language (Spanish or English) and the effectiveness of graphic organizers in computer-based instruction by 
comparing mean gain scores across language groups. To accomplish this assessment, the experiment was 
structured so that gains in academic achievement scores of two groups of fo'trth-grade students after they 
engaged in computer-based instruction could be compared. One group accessed an instructional program 
with embedded graphic organizers; the second group accessed an analogous instractional program, but witii 
embedded lists of topics. The term embedded refers to features contained witiiin the programs. Gain score 
differences between pretest and posttests (immediate and delayed) weie used to determine whetiier the type of 
embedded feature resulted in a difference in short-tenn and long-term recall. 



Methods 



Population and Sample 

The population under study was fourth-grade students engaged in computer-based instraction. Students 
enrolled at a single elementary school in a suburb of San Diego, California, served as tiie accessible 
population. The sample used in this study was 68 fourth-grade students. Two fourth-grade classes with 
different teachers were included. The sample included 31 English speaking and 37 Spanish speaking 
students. English and Spanish speakers were defined by each individual's dominant language of instruction 
according to school authorities. We dropped three students from the study: Two students did not submit 
pretests and a tiiird did not submit the delayed posttest. Each subject who completed all components of this 
study (n = 65), participated in a pretest, instructional program, immediate posttest (Posttest 1), and delayed 
posttest (Posttest 2). 

Two important variables led us to choose the location for our study. First, 58% of the school's students are 
classified as Limited English Proficiency (LEP) or Non-English Language Background (NELB) students. As 
such, this district represents a microcosm of demographic changes that will soon be facing many California 
and United States schools in the near tiiture as enrollment of sheltered English students increases. The LEP 
and NELB subjects in this school are almost all Spanish speakers with Mexican cultural backgrounds. 
Second, all students between second-grade and fifth-grade spend approximately 20 minutes per day engaged 
in computer-based instruction. Because of this exposure, and because the study involved computer-based 
instruction, test results should not be contaminated by the novelty effect that may have influenced results at 
other research sites. 



Malsnals 

Instructional programs . Tlie instructional material included four versions of a computer-based 
program developed using IBM's "LinkWaylM" software. The programs were designed to include content 
normally presented in the school's fourth-grade science curriculum and structured so that students could 
complete the instruction in 20 to 30 minutes. 

We developed four analogous versions of the instructional program. Two versions were developed in 
English and two in Spanish. In each language, we developed one program with embedded graphic organizers 
and one with embedded lists of topics. The difference between the programs was the position of concept 
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labels on the screen and the use of line, connecting these concept labels in the graphic organizers. 
Otherwise, the programs were identical. Figure 2 shows partial renditions of screens depicting this 
difference. 
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Figure 2 . Renditions of lists-of-topics and graphic-organizer screens. 

The programs consisted of 21 screens containing instructional information, 7 screens representing either 
lists of topics or graphic organizers, and 3 screens containing user directions and general information 
regarding the program's presentation. 

Screens that presented lists of topics and graphic organizers were designed with three colors. Green 
identified what material users would sec next in each treatment. Pink identified what was just viewed, and 
white indicated the overall content of material. The colors used in the list-of-topics and graphic-organizer 
screens served four purposes: 

to provide feedback to the learners on their progress through the program; 

• to serve as a preview strategy, highlighting material to be learned (green); 

• to serve as a review strategy, highlighting material just learned (pink); and 

• to serve as a synthesized overview of all the information in the program (white). 

Test Instruments. We developed two versions of the test to match the dominant language (English 
or Spanish) of the students. The pencil-and-papcr academic achievement test consisted of 18 multiple choice 
questions from the facts, concepts, and rule/principle levels of Gagn6's (1977) taxonomy of cognitive 
skills. Teachers at the treatment school conducted a face-validity measure of the test instrument prior to its 
use with students. We used the test to gather pretest, immediate posttest, and delayed posttest scores. 
Treatment 

A stratified sample of dominant English and dominant Spanish speakers were randomly assigned to two 
treatment groups: Treatment 1 students accessed graphic-organizer screens in the instructional program; 
Treatment 2 students accessed list-of-topics screens in an otherwise identical version of the program. Within 
both treatment groups, students accessed instruction developed in their dominant language. Dominant 
language (English and Spanish) and program type (graphic organizer and list of topics) served as independent 
variables. Mean gain score differences of achievement tests served as the dependent variables for group 
results. 

The instructional program was limited to a one-shot, 20 minute treatment. All students were provided with 
a later opportunity to use the non-treatment program to ensure equal educational opportunities to all 
subjects. 
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The teachers' role was limited to administering the pretests, immediate posttest, and delayed posttest, and 
directing students to the computer where they accessed their pre-assignwi instructional program. 
The administration procedures for this study involved four steps. First, all students took the academic 
pretest matched to each individual's dominant language. Approxim ^ly one day later, students began using 
the treatment program. Because each classroom had only one available computer, students took turns using 
the program. After completing the program, students took the immediate posttest to measure their short- 
term recall. Approximately two weeks after the treatment students took delayed posttest to measure their 
long-term recall. 

Exi?crimgntal Dgsign 

An analysis of variance (ANOVA) statistical procedure was used to identify whether statistically significant 
differences existed between group mean gain scores. Gain scores were defined as the difference between 
pretest and immediate posttest scores (Gain 1), and the difference between pretest and delayed posttest scores 
(Gain 2). Gain scores were used to avoid the impact of individual differences on the level of content 
knowledge at the start of the study. Dominant language and program type were used as independent 
variables. Gain 1 and Gain 2 scores served as dependent variables. 

We used a one-way ANOVA to determine if statistically significant differences existed between the means of 
the two treatment groups. To identify if statistically significance differences existed among the four 
subgroups (two languages by two treatments) a two-way ANOVA was conducted. A predetermined overall 
level of statistical significance was set at 0.05. The prcdetemiincd level of educational significance, as 
measured by the standardized mean difference effect size (calculated by dividing mean differences of the 
posttest by the standard deviations of the scores from the untreated groups) was set at 0.25 (Tallmadge, 
1977). 

Results 

Overview of Descriptive Statistics 

Table 1 presents mean gain scores, standard deviations, and sample sizes for the four treatment groups 
involved in this study. Mean gain scores are presented for two instances: Gain 1 (defined as the difference 
between immediate posttest scores and pretest scores) and Gain 2 (defined as the difference between delayed 
posttest scores and pretest scores). 

Table 1 

Means, Standard Deviations, and Sample Sizes for All Groups 



Gain 1 Gain 2 



Treatment 


Groups 


M 


SD 


M 


SD 


n 


List of Topics 


English 


2.61 


1.56 


1.01 


1.04 


13 


List of Topics 


Spanish 


3.58 


2.17 


1.63 


1.85 


19 


Subtotal 




3.19 


1.94 


1.14 


1.70 


32 


Graphic Organizer 


English 


4.20 


2.27 


2.13 


1.64 


15 


Graphic Organizer 


Spanish 


4.17 


1.95 


2.61 


2.03 


18 


Subtotal 




4.18 


2.07 


2.39 


1.71 


33 


Total 




3.69 


2.02 


1.90 


0.20 


65 


Subtotal 


Spanish 


3.87 


2.06 


2.12 


1.94 


37 


Subtotal 


English 


3.42 


1.91 


1.62 


1.30 


28 



Information in Table 1 reveals that Gain 2 scores were lower than Gain 1 scores for all treatment groups. 
The reduction in gain scores reflects an expected decrease in the amount of information retained over time. 
Initial analysis indicated that in immediate and delayed recall tests, mean gain scores of both English and 
Spanish speaking groups using programs with embedded graphic organizers were higher than those achieved 
by their counterparts using programs with embedded lists of topics. These results lead us to believe, at least 
with descriptive statistics, that embedding graphic organizers in instructional programs is a better strategy 
to improve academic gains than embedding lists of topics in instructional programs for both Spanish and 
English speaking fourth-grade students. 
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Statistical and Educational Significance 

Graphic organirArs vrrsiis list nf topics . From data in Table 1, the standardized mean difference 
effect size between groups was calculated using computer-based instructional programs with embedded 
graphic organizers and emDedded list of topics. Effect sizes were identified to be 0.40 for Gain 1 scores and 
0.5 1 for Gain 2 scores. An ANOVA was conducted for both Gain 1 and Gain 2 scores (see Table 2). 



Table 2 

Analysis of Variance: Comparison of Gain 1 and Gain 2 for English and Spanish Speakers Grouped by 
Program Type 



Dependent 
Variable 


Source 


Degrees of 
Freedom 


Sum of 
Squares 


Mean 
Square 


Ftest 


Significance of 
one-tail F test 


Gain 1 


Between groups 


1 


16.06 


16.06 


3.99 


p = .04 




Within groups 


63 


253.78 


4.03 








Total 


64 


269.84 








Gain 2 


Between groups 
Within groups 
Total 


1 

63 
64 


15.84 

187.59 

203.44 


15.85 
2.98 


5.32 


p= .01 



The information in Table 2 indicates that there was a statistically significant difference between treatment 
groups favoring graphic organizers on both short-term (Gain 1 scores) and long-term recall (Gain 2 scores). 
Previous researchers, as discussed earlier, indicated that graphic organizers within instructional material 
improved short-term and long-term information recall. Oxir findings support these conclusions. This 
experiment also extended research into the realm of computer-based instruction and elementary students. 
Evidence now exists which supports the idea that the use of graphic organizers in computer-based 
instructional programs has a significant positive effect on both short-term and long-term recall for fourth- 
grade students. 

Dominant language differences . A two-way ANOVA was used to determine if the effectiveness of 
graphic organizers in computer-based instructional programs varied significantly by learner's dominant 
language. In this computation, program type and dominant language were used as independent variables and 
gain scores as the dependent variable. This analysis is sho wn in Table 3. 



Table 3 

Analysis of Variance: Comparison of Mean Scores for Graphic Organizer Users Grouped by Dominant 
Language 



Dq)endent Source 


Degrees of 


Sum of 


Mean 


Ftest 


Significance of 


Variable 


Freedom 


Squares 


Square 




one 


-tail F test 


Gain 1 Program 


1 


3.44 


3.44 


0.85 


P = 


.03 


Language 


1 


18.74 


18.74 


4.64 


P = 


.36 


Program X Language 


1 


3.95 


3.95 


0.97 


P = 


.33 


Error 


61 


246.61 


4.04 








Gain 2 Program 


1 


16.46 


16.46 


5.48 


P = 


.02 


Language 


1 


4.23 


4.23 


1.41 


P = 


.24 


Program X Language 


1 


0.02 


0.02 


0.01 


P = 


.93 


Error 


61 


183.36 











Information presented in Table 3 indicates that there is no statistically significant difference at the 0.05 
confidence level between language groups for the effect of graphic organizers in immediate (Gain 1) or 
delayed tests (Gain 2). These results indicate that the effectiveness of graphic organizers in computer-based 
instructional programs does not vary significantly by learner's dominant language. The standardized mean 
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difference effect size calculated from Table 1 was 0.01 for Gain 1 scores and 0.19 for Gain 2 scores. These 
scores indicate that differences on the effect of graphic organizers between language groups are not 
educationally significant for either short-term or long-term recall. 

Discussion 

Implications of the Stu dy 

Two main implications can be deducted from the results of this study. First, the use of programs 
with embedded graphic organizers in computer-based instruction enhances short-term and long-term recall in 
fourth-grade students. The increase in academic achievement scores measuring long-term and short-term 
recall confirmed previous research findings. Second, results indicate that dominant language differences do 
not significantly impact the effectiveness of programs with embedded graphic organizers. The non- 
significant differences between dominant Spanish and dominant English speaking students were unexpected. 

Researchers agree that the effectiveness of any learning strategy depends on context and content of 
instruction, and on individual learner's experience, cosnitive maturity, motivation, and learning style 
(Oxford, 1990; Schmcck, 1983). In this study, context ai.J content of instruction were analogous for all 
treatment groups. One assumption made was that subjects did not substantially differ in terms of their 
experience and cognitive maturity because they were at the same grade level, they were randonJy assigned to 
treatment groups, and their pretest scores did not indicate statistically significant differences at the 0.05 level 
(Gimenez, 1994). Therefore, our results indicated that when using computer-based instructional programs 
with embedded graphic organizers, Spanish speaking and English speaking students, as groups, did not vary 
on leaning style to the point of significantly impacting academic achievement scores according to tests used 
in this study to measure short-term and long-term recall. 

Theoreticians suggest that cultural background provides the most important resouices for individual 
cognitive development (Connolly & Tucker, 1982; Stein, 1990; Sticht, 1992). Such resources refer to 
symbols and symbol systems, including natural language and conceptual knowledge. These are the primary 
tools for the transmission of cognitive abilities and motivational conditions (Sticht, 1992). Results of this 
study indicated that the different cultural backgrounds of dominant Spanish and dominant English speaking 
subjects did not significantly influence the effectiveness of graphic organizers embedded in computer-based 
instructional programs. 

In searching for causes of academic achievement differences between dominant English and 
dominant Spanish speaking students, researchers determined differences in learning style preferences as key 
elements (Scarpaci & Fradd, 1985; Schaiper & Flores, 1985). These researchers suggested that culture 
strongly influences the way in which students learn and interact Instead of faulting individual 
characteristics, studies on LEP and NELB native Spanish students identified that low scholastic achievement 
was related to instructional materials, instructional methods, classroom environments, and learning 
strategies that favor native English students (Dunn, 1990; Ulibarri, 1982). Recent studies determined that 
the United States educational system, at all levels of instruction, relies on the learning preferences of white 
middle-class students, and these preferences are not always shared by Mexican-Americans (Cohen, 1969; 
Connolly & Tucker, 1982). However, results of this study indicated that graphic organizers embedded in 
computer-based inctructional programs significantly increased mean academic achievement scores of both 
groups, and did not significantly vary between dominant English and dominant Spanish speaking students. 

Results of this study support previous theories suggesting that the use of graphic organizers is an 
effective strategy to increase short-term and long-term information recall. Although both groups had a 
decrease in Gain 2 scores reflecting long-term recall, this decrease was more pronounced for the group using 
programs with embedded list of topics. These results suggest that the use of computer-based instructional 
programs with embedded graphic organizers have tlieir greatest impact on long-term recall. Although the 
study was limited to a one-shot 20 minute treatment, the results were significant. These findings, concluded 
from a single, short exposure to graphic organizers, foretell of potentially major cognitive growth in 
students who use this strategy for extended periods of time. Thus, educators and instructional designers can 
strongly encourage the use of embedded graphic organizers in instructional materials as means to enhance 
short-term and long-term recall of information. 

Limitations of th? Stutjy 

Given the characteristics of the study, several shortcomings are apparent. First, teachers did not monitor 
students during the treatment. It is possible that students did not complete the treatment or may have used 
the program assigned to a different group. However, data were consistent with previous studies. This 
consistency lends support to our conclusions. Second, the treatment was limited to a one-shot, 20-minute, 
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computer-based instructional program. Although this short exposure was initially considered a weakness in 
our study design, it may also be viewed as supporting the strength of graphic organizers. Because students 
exhibited significant short- and long-term gains after a brief exposure, longer exposures to graphic 
organizers may generate even greater gains in academic scores. 

Questions for Future Reseaish 

Questions and concerns for future research include the following: 

1 . Do graphic organizers embedded in computer-based instructional programs improve academic 
achievement scores when the graphics are used as navigation controls? This study focused on the use of 
graphic organizers as graphic representations of information. However, literature reveals that graphic 
organizers in computer-based instrucdon may also serve as navigational tools (Barba, 1993). The literature 
on leamer control, however, is controversial. Some researchers suggest that because graphic organizers may 
be used aj* a nonlinear presentation controller, the impact on the instructional process may vary according to 
die learner's cognitive maturity, previous knowledge, learning style, and preferred leaning strategies. 
Researchers determined that although learner control is motivating, the lack of structure in the instructional 
sequence may cause oarriers for some learners (Malone & Leppcr, 1987). 

2. Does the use of graphic organizers in computer-based instruction improve academic achievement 
scores when the organizers do not provide feedback on leamer progress? Although this study focused on the 
use of graphic organizers as graphic representations of information, these organizers also served to provide 
feedback on leamer progress. Feedback was provided by using colors to identify which concepts had just 
been explained, which were about to be viewed, and die overall structure of the lesson. Future research may 
replicate this study without the use of feedback to detennine whether this feature was a critical variable for 
the effectiveness of graphic organizers in this study. 

3. Docs the use of grapliic organizers in computer-based instruction improve academic achievement 
scores when they include labeled links? Explicitly labeling the relationships in a graphic organizer may 
cause information overload in students widi lower cognitive capabilities. In tiiis study, labeled relationships 
were avoided because it was assumed Uiat they may impose extra cognitive processing leading to 
information overload for fourth-grade students. Researchers may want to compare die effectiveness of 
graphic organizers with and witiiout labeled relationships with the population used in tiiis study. 

4. What are the characteristics of students who do not improve academic scores when they use 
computer-based instructional programs with embedded graphic organizers? A further study could identify 
characteristics of students who prefer to use lists of topics to graphic organizers by exposing learners to 
both strategies and allowing them to choose the strategy that has the greatest impact on their short-term and 
long-term recall (measured in terms of academic achievement scores). By identifying attiibutes of both 
groups of students, instruction could be tailored to better meet individual needs. 

5. This study focused on Spanish and English speakers who are in a bilingual environment. 
Researchers may wish to replicate tiiis study to include subjects in monolingual educational settings, and in 
settings with other than Spanish and English speakers. 
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